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ABSTRACT 

The Defense Communication System will be moving rapidly toward providing 
switched digital service to it's users within the next ten years. The principal 
driving force in the transition to a digital system is the requirement for high 
performance secure voice service. Additionally, the anticipated data require- 
ments in this time frame can be handled most effectively by a digital network. 
The characteristics of a switched digital network which impose timing and syn- 
chronization requirements on the system design will be presented. 

Several alternative approaches to implementing a timing subsystem suitable for 
a switched digital communications system have been considered. These include 
pulse stuffing, independent stable clocks, and clock correction techniques. The 
advantages and disadvantages of each approach will be discussed relative to both 
the strategic and tactical communication system requirements. 

An inter-agency committee worked through mid-1973 to develop recommended 
parameters for interoperability between the various DoD communications sys- 
tems, allied systems, and the commercial networks. 


BACKGROUND 

Synchronization in a communications system can be considered on at least three 
different levels— terminal-to- terminal, link, and network. Terminal-to- terminal 
synchronization is widely used today for digital communications between data 
terminals using modems connected by circuit- switched or dedicated analog chan- 
nels. The terminal device, or the transmit portion of the modem, provides 
transmit clock. The receive modem develops timing from the incoming data 
stream (transmitted as quasi-analog signals through the analog channel), and 
detects and regenerates the digital data. Frame synchronization is developed 
by the terminal equipment as required. Thus the terminals, consisting of digi- 
tal devices and modems, are designed to operate over half-duplex or full-duplex 
circuits, and develop timing and synchronization between the calling and called 
terminals independent of the network. 





***** 


529 



Link synchronization is used in digital multiplexing, such as in the commercial 
T-type PCM carrier equipment. In the primary channel bank a group of analog 
channels (24) are digitized and combined into a single digital stream. Timing is 
supplied by a clock internal to the channel bank. The digital stream is either 
transmitted to the distant link terminal via cable or radio, or is combined with 
several other similar streams in a second level time division multiplexer (TDM). 
Since each channel bank has an independently derived time base, the TDM must 
asynchronously combine the various inputs to obtain a single higher rate output. 
The pulse-stuffing technique is used, whereby pulses are added to each input as 
needed, to bring all inputs to a single common higher rate. These added, or 
"stuff”, bits are deleted at the receive terminal using information carried on a 
control channel associated with the second level multiplexer. In this case, as 
in the terminal-to- terminal case, timing and synchronization is provided between 
transmit and receive channel banks. The network has no synchronization re- 
quirements, The individual channels need not be synchronized with one another 
because the digitized multiplexed channels are returned to their analog form 
(which does not require timing) at each switch. A circuit may be digitized a 
second time at the output of a switch, but it will be to a new time base established 
by another PCM channel bank. 

Network timing and synchronization must be considered when digital multiplexed 
channels are to be switched. This follows from the requirement for many dif- 
ferent digital channels originating at various points in the network to be simul- 
taneously passing through the switch with varying connectivity. If the switch uses 
time division switching, all these channels must be operating on the same time 
base as the switch. Even if a space division switch is used, where common 
timing is not required in the switch, common timing will nevertheless be required 
at the multiplexing point. That is, if the channel carrying digital signals from 
device "A" is to be multiplexed with the channel servicing device "B", these 
channels must be synchronous. 

Precise network synchronization is fundamental to maintaining bit integrity in a 
digital switched system. Even when high error rates are experienced on the 
transmission media, or when the multiplex equipment loses framing and generates 
an error burst, if the time base is accurately established between the two nodes 
the system can maintain synchronization through the link interruption. This 
justifies a precise synchronization specification. 


NETWORK SYNCHRONIZATION CONSIDERATIONS 

There are two principal approaches to achieving network synchronism— (1) oper- 
ate all terminals independently and use the pulse stuffing technique to align the 
channels at the switches aad multiplex, and (2) develop a form of synchronous 
network such that all equipment is operating at a constant frequency or time base. 


530 



Although the channel pulse-stuffing technique is technically sound, and offers 
the possible advantage of avoiding network synchronization problems, the tre- 
mendous proliferation of stuff-destuff devices and attendant cost, reliability and 
maintenance problems, etc. , act to negate this advantage. These devices would 
be required at every digital channel terminal, at every channel termination on 
a digital switch, and at every channel input to a multiplexer. No digital network 
now in the planning stages is known to be based on Individual channel pluse- 
stuffing. Thus only the alternative of a synchronous network is considered fur- 
ther in this paper. 

The definition of "synchronous network" can become highly complex and unen- 
lightening. From a practical viewpoint, the principal parameter is the period of 
time over which synchronism is maintained. For example, a pair of intercon- 
nected Teletype machines are synchronous for the period of one character. Sim- 
ilarly, an independent clock technique is described later which provides network 
synchronism for periods on the order of one day. The objective for the Defense 
Communications System (DCS) is network synchronism for an unbounded period 
of time, which requires the same average frequency at all nodes of the network 
over some specified period of time or equivalently, the maintenance of a time 
coherence with some fixed, specified tolerance between all nodes of the network. 

In addition to the network timing system which provides the time and/or frequency 
base throughout the system, "elastic" buffers are required at the termination of 
each transmission link at a network node. The purpose of this buffer is twofold: 
(1) to provide a reservoir of bits to adjust for instantaneous differences in the 
transmitting and receiving rates at the two ends of the link, and (2) to adjust for 
variations in the absolute time delay through the link. The size of the buffer 
will depend on the magnitude of these two factors, and on the digital rate of the 
link. Thus clock tolerance and buffer size, adjusted by path delay variations, 
can be traded to obtain the most effective system design. 

A number of network synchronization techniques are capable of satisfying the 
basic communications requirement for keeping bits in the proper sequence at 
time division multiplexers and/or switches. These include such concepts as: 

(1) timing derived from external sources such as the Loran-C navigation sys- 
tem, (2) an independent clock system in which precise clocks are placed at 
each node, (3) discrete control correction (a form of frequency averaging) in 
which a weighted average of the contents of the storage buffers at each node are 
used to correct the frequency of the nodal clock at discrete intervals of time to 
prevent buffer overflow, and (4) time reference distribution in which time re- 
ference information is distributed over every transmission link of the network 
to assure that all nodal clocks of the network have the same time. 
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One presently planned communication system (TRI-TAC) will use the independent 
clock system with a tolerance on frequency rather than time. The principal ad- 
vantage of this technique is that the timing of each node is self sufficient and inde- 
pendent of the rest of the network, which is a highly desirable property for a 
tactical, mobile system. Atomic standards will be employed at each node to 
provide a frequency accuracy on the order of one part in 10 11 . This accuracy 
permits the use of reasonable size buffers to equalize the frequency difference 
between nodes for periods up to 24 hours. The buffers are reset, with attended 
link interruption, when required. The impact of link interruption for buffer re- 
set is considered undesirable, and it is not acceptable for a strategic world-wide 
communicatk a system. 

A different technique, which avoids the buffer reset requirement, obtains the 
time reference for the nodal clocks from a source external to the communications 
system. Several sources could be used such as Loran or navigation/timing satel- 
lites. The use of Loran-C to provide the reference imposes the least technical 
risk of all the different techniques. It can be made compatible with other systems 
if satisfactory standards and buffers are provided. Its accurate time can be 
used for the resynchronization of switches, multiplex, and cryptographic equip- 
ment. It is an existing technique for which considerable experience has been ac- 
cumulated. It could be implemented with a high degree of confidence. However, 
the survivability of a communication system based on an external reference sys- 
tem must be considered, since the required use of an external reference system 
for the DCS would make that timing system a "desired” target. Also Loran-C 
presently does not provide world-wide availability, thereby limiting its application. 

The discrete control correction system uses the weighted average of the contents 
of all of the buffers at each node to make frequency corrections to that nodal 
clock at discrete instants of time. The timing at the nodes is independent except 
at these discrete correction times. Studies have shown that this technique pro- 
vides a stable network and frequency base, and that its frequency can be locked 
to that of an external network by selecting the proper weighting factors. Its ma- 
jor advantages are simplicity of implementation and the lack of any requirement 
for resetting of the data storage buffers. 

The time reference distribution system might be considered to be an upgrading 
of the TRI-TAC independent clock technique, wherein the clock time at each 
node is corrected instead of resetting the buffers. In this concept the communi- 
cations path is used for the distribution of a time reference. This reference is 
based on the master clock in the system; e.g. , the Naval Observatory. All nodes 
have their time traceable back to this master. If the master is lost, provision is 
made to pass this function to another "qualified" node in the system. When con- 
sidered from an overall system point oi view, including system monitoring, 
maintenance, trouble shooting, and the application of future technology, it has 
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many advantages. Since a time reference is available from every incoming link 
at a node, each node will always have a time reference if it is connected to the 
network. Thus the technique is highly survivable. Its major disadvantage is 
that it is somewhat more complex than other alternatives and has not been proven 
in practice. However, its concepts are closely enough related to systems which 
have been or are being implemented so that it can be approached on an evolution- 
ary basis with a high degree of confidence. Refer to "A Time Reference Distri- 
bution Concept for a Time Division Communication Network" for a more detailed 
discussion of this technique. 


DOD INTEROPERABILITY COMMITTEE RECOMMENDATIONS 

Interoperability among the DCS, tactical systems, foreign allies, and domestic 
and foreign commercial systems is deemed an essential requirement in the de- 
sign of future systems. Common, or interoperable timing and synchronization 
between these systems is a basic parameter in achieving this requirement. To 
address this, and the many other parameters involved in interoperability, a joint 
committee was formed operating under the co-chairmanship of DC A, TRI-TAC, 
and NSA. Also participating were the representatives of the military services 
and the OJCS. The report of this committee, "Final Report of Committee on 
Interoperability of DoD Telecommunications", dated 20 September 1973, contains 
recommendations for common or compatible values, methods, and procedures 
for the system parameters considered to have an impact on the ability to provide 
end-to-end encryption of DoD telecommunications systems. Specific recommen- 
dations are made for network timing and synchronization. 

Several forcing factors drive the selection of the time/frequency nodal tolerances 
for the DoD communications systems. These include the considerations for tac- 
tical applications, performance suitable for the long-haul DCS, and the need to 
maintain bit-count integrity for satisfactory overall digital system performance. 
Tactical applications involve optimizing parameters for simplicity, survivability, 
flexibility, ease of movement, and reconstitution of network operations. In the 
light of these requirements, an independent clock system with frequency tolerances 
was specified as best meeting these tactical requirements. However, buffers 
must be reset on occasion, causing traffic interruptions. Since interruptions to 
the DCS carry a much greater impact, such a system in unacceptable for the DCS 
unless other approaches cannot be proven or are substantially more costly. 

Recognizing the near-term i»?ed for digital systems, the committee recommended 
that equipment under current development continue to utilize the independent clock 
approach designed to meet the specification tolerances of TRI-TAC (Table 1). 

This frequency tolerance (one part in 10 11 per day), combined with properly sized 
buffers, will limit buffer reset to at most once per day when frequency standards 
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Table 1 

TRI-TAC Timing Specifications 


Frequency Tolerances 


Primary STD 

Secondary STD 

Backup 

1 x 10 *" /day 

1 x 10' ,l /day 

1 x 10^ /day 

1 x 10'"/6 mos. 

1 x 10 " lw /6 mos. 


Setting Accuracy 

Repeatibility 

Primary STD 

Secondary STD 

Secondary STD 

1 x 10 n 

1 x 10 11 

3 x 10~" 


at adjacent nodes are far from their nominal values in opposite directions. Such 
current development must not preclude the future inclusion of some form of net- 
work clock correction technique to provide the future capability for long-term 
network synchronization, fail-safe operation in the equipment failure mode (that 
is, utilization of the independent clock system in a fall-back mode), and avoid- 
ance of the need to reset buffers. 

As an objective for the ultimate DoD, a time tolerance of ±2 microseconds be- 
tween any two major nodes was recommended. Minor nodes, such as PABX's, 
could be slaved to major nodes and must provide independent timing to their sub- 
scribers whenever such minor nodes become isolated from the network. 

The TRI-TAC timing requirements will be met through the use of atomic fre- 
quency standards. Tactical atomic frequency standards are proven and their 
costs, including the required buffers, are a small fraction of the nodal costs. 
Their performance is two orders of magnitude better than currently planned 
commercial standards. Therefore, no difficulty with commercial interfaces is 
expected in this respect. As techniques are proven and implemented to attain 
the ±2 microsecond time tolerance objective for the ultimate system, both fre- 
quency standard and the time standard can coexist in the network without diffi- 
culty or constraint. 


CONCLUSIONS 

The adoption of the recommended frequency standard for the implementation of 
near-term DoD systems will result in garbling a relatively small amount of 
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traffic when buffers are reset. While this standard is in use, care should be 
taken in network operation to minimize this effect. Such care includes: reset- 
ting whenever traffic is interrupted for other purposes such as crypto key change 
on the link; preplanned resetting at low traffic periods; and off-loading of switched 
traffic prior to resetting. 

The recommended objective for future DoD systems requires time coherence to 
a specified tolerance throughout the network. Although time coordination among 
file many nodes of a large communications network has not been accomplished, 
fiie basic concepts involved have been exercised in the coordination of such Navi- 
gation networks as Omega and Loran-C, and through the DSCS time transfer 
program. Thus, it is believed t^at a long-term coherence between nodes of a 
network is a feasible and useful objective for communications purposes, and will 
provide a precise source of time to other users as well. 
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QUESTION AND ANSWER PERIOD 


DR. REDER: 

Did I understand you right that you said the TRITAC committee recommended the 
use of the atomic standards with the provisions that they are not supposed to be 
reset to the standard in frequency more often than once a month— once in six 
months ? 

MR. MENSCH: 

As I understand it, if rubidium standards were used, they would have to be reset 
in frequency occasionally. If they were used— there is a contractor option. The 
contractor chose to go cesium, there is no reset requirement in that type of 
clock, as I understand it. 

DR. REDER: 

This is my question. Is there no requirement? I think there is. If you do not 
allow for resetting the crystal, for instance, in the 5061, you may be in trouble, 
if the requirement is 1 part in 10 to the 11th. 

MR. MENSCH: 

Will that cesium clock stay within absolute center accuracy, to within 1 part in 
10 to the 11th? 

DR. REDER: 

I have my doubts. 

DR. WINKLER: 

I think there is a misunderstanding here. 

So far as I understand your requirements, Mr. Mensch, the emphasis is on ex- 
ternal reference. In other words, this, what you are referring to, would have 
to be incorporated in the maintenance instructions for that cesium standard. It 
is true that the cesium standard if left all by itself for one-half year would not 
necessarily stay within 1 part in 10 to the 11th, but that can be taken care of 
following the instruction manuals in reference to the cesium standard itself. 

That is, in my opinion, fundamentally different from the situation with the rubid- 
ium standard where you will have to make even larger adjustments, with respect 
to an external reference. 


536 



MR. MENSCH: 


This is my understanding also. Or. Winkler, that in the case of rubidium, some- 
thing, like the traveling clock, or some external means would have to be brought 
in to that terminal in order to bring that within the tolerance window that would 
be allowed. 

DR. REDER: 

Apparently I took too literally what you said. If you permit maintenance, which 
means you watch the meter, and if the error meter shows a deviation more than 
a certain amount, that you are permitted to reset it, fine. 

I thought maybe there was somebody standing with a fly swatter and you know, 
knock you on the hand or something if you touch it. 

(Laughter.) 

DR. KARTASCHOFF: 

May I perhaps add to this discussion by saying that the current view is in my 
country now, my own and some of my colleagues, about the use of rubidium 
versus cesium ? We intend to use rubidium on the relatively low level , low rank 
exchange— not the lowest, but low rank exchanges, and we will slave them to 
cesium standards in the country. 

You just have to look at the results in the U. S. Naval Observatory— they kept a 
part in 10 to the 12th over six months. 

MR. MENSCH: 

This would be in more of a synchronized network. Now, in the TRITAC thing I 
am referring to, let's assume that they implement the equipment that is used at 
a node with a rubidium standard. These would be sprinkled throughout a net- 
work. There would be no synchronization or adjustments between them. There 
would just be buffers between them, and those buffers are large enough to soak 
up that frequency difference over some period of time, like 24 hours. 

If that buffer fills in that period of time, you then push the button, throw some 
bits away, and start over. They will also automatically push that button if the 
link fails between them, if you have a fade. There is another aspect of this, 
that this link will be inerypted, and periodically we have to come in and change 
the variable in that crypto equipment, at which time again we drop sync. 
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So, it is not just due to frequency errors. Will vs have an opportunity to, lets 
say, not just to buffer fill or empty, will we have a situation where we have a 
loss of sync and get to reset the buffer. 

But the point is that each of these nodes in the network is free- running, but they 
are synch rone us for a finite period of time. 

Now, we in the long haul business, let's say, of the Defense Communications 
System, do not want to design a system that requires us to drop sync on purpose, 
so we are staying on top of that now, let us correct those clocks by some means, 
either through external timing system, or by distributing time through our net- 
work, and as long as we keep those clocks corrected everybody is happy. If we 
do lose that system, we will then fall back in a fail safe mode to running inde- 
pendently over a short period of time, as short as 24 hours. 

Now, the interesting aspect, if we look at the impact of the satellite systems in 
such a communications network, since satellites will be— are becoming, if we 
ever get some satellites up that work, a very important mode of communications. 
The potential variation in that path, depending on the stability of the orbit itself, 
but we could end up with hundreds of thousands of bit buffers to soak up the abso- 
lute path delay variations of a satellite link, if we are working on the order of, 
let's say, a megabit or two, of our communications link. And we do have some 
orbital perturbations in that bird. 

We are talking about fairly sizeable buffers, and are accepting the fact that in a 
synchronous network this must be. Now that will be a special box in the satel- 
lite link, as opposed to the land situation. 

DR. WINKLER: 

May I make another comment here, not a question but a comment. 

It is in regard to the frequent reference to a master reference standard— pre- 
sumably they use the U.S. Naval Observatory's master clock. 

In view, however, of any of these systems really being totally worldwide, the 
question of possible interface with NATO or other allies inevitably arises. And 
here, I would like to Inform you that, for those who do not follow the time scale 
bulletins regularly, that we have demonstrated the capability during the last 
seven months or so, to stay within one-tenth of a microsecond of the interna- 
tional time reference. 
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It is our belief that even with very small variations in IAT, the international 
reference time, that we can sufficiently well in advance predict these changes 
to stay safely within one microsecond. Actually, we have done ten times better. 

So, giving you a number of two microseconds, to which you want to keep all these 
clocks, I think that you have already a system available in which you have access 
to any of the national references in the international system, and for which cor- 
rections to 0. 1 microseconds are known. 

MR. MENSCH: 

You are saying you are demonstrating the feasibility of being able to achieve this. 
DR. WINKLER: 

Yes, These numbers are published in advance, in fact. We extrapolate the dif- 
ference between the master clock and the international BIH time, and if you look 
at the BIH bulletin Series D, which we distribute upon request, if you look through 
these you will find that we only have random fluctuations around zero, minus 
one- tenth, plus one-tenth around zero. It can be done. I am quite confident that 
we will continue to do so in the future. 

MR. MENSCH: 

Dr. Winkler, you made one point that I had a note that I didn't mention, and it 
comes up this way, that we will also have a requirement to interface our digital 
network with commercial people. 

We are encouraging, and plead with anybody that is here, that the commercial 
systems also consider operating off the same master clock, for international 
time, to ease the problem of when we hook our channels to them, if they are run- 
ning on a separate time base, we are going to have to have an awfully big buffer, 
or reset, and this becomes a problem, because the majority of the Defense 
Communications Systems is leased channels to the common carriers. We do not 
own the majority of our circuits. 

We are hoping these common carriers, such as the Bell Digital Data system, 

DA TRAC, NCI, et cetera, will go toward not only having master slaves, but that 
their master clc ~k becomes time coherent with that of our master reference. 
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DR. WINKLER: 


Yes. There is yet another problem which has to be addressed, or not forgotten, 
and that is the establishment of a well-defined hierarchy in the interest of sur- 
vivability. 

MR. MENSCH: 

This is a point that I think was made in Stover's paper. These rules provide 
exactly this, that there is built into that type of a system, or a master slave 
system, a procedure that if you lose your master clock, you have a graceful 
passing down of the master responsibility. 

DR. WINKLER: 

And that is exactly why we want to use precision clocks, and that is where the 
difference between a synchronized system and a coordinated system is most 
striking. 

A synchronized system does not operate with any inertia. However, a coordi- 
nated system, once you have all your clocks in operation for a couple of weeks, 
you can actually completely cut your communications, as we have done in other 
systems already, and you will still stay within a microsecond for a considerable 
period of time. 

MR. EASTON: 

We have time for one more question ? 

MR. CHI: 

In view of your stated requirement of a part in the 11th per day, and plus or 
minus two microseconds, and especially if you use a hierarchy does it not mean 
also that almost any kind of standard with reasonable stability, such as a crystal, 
should also be considered. It is lust a matter of tradeoff, with the frequency of 
correction as well as the costs. 

MR. MENSCH: 

And the size of the buffers that are to be used. It is a tradeoff. 

So, the tactical people are going to this atomic clock version such that they can 
have their little node, or their communications node, and put them and move 
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them several times a day, and not have the network synchronization problem. 
They just turn it on and they are all set. 

This is the tactical doctrine. 

In the DCS, where wedonothave the mobility requirement, we can see a sprink- 
ling of precise clocks, be they atomic or disciplined, or however we achieve it. 
But that it is coordinated, as Dr. Winkler says, that they may very well be. The 
majority of our nodes may be disciplined quartz oscillators, for example, from 
the cost effective standpoint. Certainly with seven hundred nodes scattered 
around tne world we do not want to have triple redundancy cesium beam stand- 
ards every place, just from a maintenance standpoint. 

But certainly at our major switching nodes it would be a good idea, since these 
would be the next step in the evolution of hierarchy. 

So, the big cost gain is yet to be done. We have not done this, although this then 
is going to be part of the choosing of what concept is used to arrive at this time 
coordination, and this is our next step of study to select and recommend the 
appropriate concept for a future system. 

MR. EASTON: 

Thank you very much, Mr. Mensch. 
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